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LN'J RODUCTOHY STATKMENT 



The Center for Social Organization of Schools has two primary 
objectives: to develop a scientific knowledge oi how scliools affect 
their students, end to use this knowledge to develop better school 
practices and organization. 

The Center works through five programs to achieve its objectives. 

Tiie Academic Cames program has developed simulation games for use in 
the classroom, and is studying the processes through which games teach 
an’ evaluating the effects of games on student learning. The Socia l 
Accounts program is examining how a sLudent's education affects his 
actual occupational attainment, and how education results in different 
vocational outcomes for blacks and whites. The Talents and Competencies 
program is studying the effects of educational experience on a wide 
range of human talents, competencies, and personal dispositions in older 
to formula te--and research- -important educational goals other than tra- 
ditional academic achievement. ihe School Organization program is cur- 
rently concerned with the effects of student" participation in social 
and educational decision-making, the structure of cunpetition and coopera- 
tion, formal reward systems, ability-grouping in schools, effects of 
sch'-ol quality, and applications of expectation theory in the schools. 

The Careers and Curricula program bases its work upon a theory of career 
development, It has developed a sel f -administered vocational guidance 
device to promote vocational development and to foster satisfying curri- 
cular decisions for high school, college, and adult populations, 

Jhis report, prepared by the Academic Cames program, presents the 
results if a study intended to determine the effectiveness of a business 
simulation for increasing students 1 knowledge of business facts and con- 
cepts and their ability to evaluate business decisions. Jhe results 
showed that the simulation increased the students* factual knowledge 
and possibly also increased their business decision-making ability. 
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ABSTRACT 



Fourteen high school students, chosen at random from a group of 
twenty-eight, spent five hours participating in a business simulation, 
after which all twenty-eight students took tests designed to measure 
their knowledge cf business facts and concepts and their ability to 
evaluate business decisions. The simulation group outperformed the 
control group on both tests, but the difference approached statistical 
significance only for the test of facts and concepts, 
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INTRODUCTION 



A claim frequently made for simulation as a teaching technique 
is that it teaches the players the cause-and-e CfecL relationships 
that operate in the situation simulated by th* game, thereby improv- 
ing the players' ability to make and evaluate decisions in that field 
A second common claim is that simulations teach facts end concepts which 
are important in the situation simulated by the game. 

At present, the first claim appears to be based more on faith than 
on research evidence. One reason for the lack of research evidence may 
be the difficulty of measuring such variables as "learning of relation- 
ships" and "decision-making ability." Fletcher and Dobbins (1970) at- 
tempted to overcome this difficulty by measuring students' ability to 
make verbal predictions of the outcones of situations like those simulated. 
The predictions were made in discussion groups of three students each. 
Judges listened to tape recordings of these discussions and scored each 
prediction on a three-point scale. Comparisons of "average quality per 
prediction" favored students who had played the simulation game over 
those who had studied the same situations by reading and discussion, 
but the difference was caused by a decrease in the scores of the control 
group, nther than by an increase in the scores of the experimental group. 
Fletcher and Dobbins concluded that "... game experiences do tend to 
improve the ability of tbe participants to make predictions, though nos L 
impressively with analogous cases, not with the one directly simulated 
by the game." t 



The experiment reported here tested learning of relationships 
and decision-making ability by using a written test which required 
the student to evaluate a set of decisions made in a hypothetical 
situation. The responses were scored according to the number of 
relevant relationships the student cited in making his decision. 

This technique is similar to that used by Fletcher and Dobbins, in 
that it requires the student to make cotrect and relevant predictions, 
however, it also requires him to see the relevance of these predictions 
to the decisions he must evaluate. Also, because the tests arc taken 
individually, each student r s score is independent of each other student's 
score (except, of course, for effects created by the experiments* treat- 
ment) , 



The second claim examined here -that simulations teach the players 
Tactual information about the rea] sitration--is supported by some re- 
search which indie. ites that simulations are about as effective as con- 
ventional classroom techniques for Leaching factual in format ion , * Our 
experiment included a test of factual information and vocatulary rele- 
vant to the situation simulated, 

2 

Venture , the simulation game used i i this experiment, simulates 
a consumer-products industry dominated by a small number of large 

*See, for example, Anderson MT/O) and King ( 1 006 ) , 

2 

Venture is available from the Troctor <!t (Iambic Company, Director 
of educational Services, P,0. box YY ) , Cincinnati, Ohio 65201. 
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companies. Kach team represents Uie tcp management of one of the large 
companies. The game covers six half-year periods of simulated time. 

Kach period consists of an "operations meeting” and a ''profits meeting.” 

At the operations meeting the p! iyers must decide how much to charge 
for their company's product, how mud to produce, whether to allocate 
funds for depreciation and foi murker research, and how much to allo- 
cate for marketing and advertising and for research and development. 

At the profits meeting the pl.-y^rs allocate their company's profits 
(after taxes) among dividend:, to st ockholders, investment to reduce costs, 
investment to increase capacity, investment to improve research and develop- 
ment facilities, and increased operating revenue. As in a real business, 
there is no single criterion for evaluating performance; each company T s 
sales, profits, and stock price are posted publicly and can all serve 
as criteria. 
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S ubjects 

The subjects for this experiment were 28 students --27 seniors and 
one junior--at Cardinal Gibbons High School, an all-boys Catholic high 
school in Baltimore • All the subjects were paid volunteers, ’Ihey were 
divided at random into an experimental and a control group of 14 stu- 
dents each f 'the experimental group spent about five hours on a Saturday 
participating in V ent ure ; the control groun received no treatment. The 
following Monday, the students in both groups were given the tests to 
fill out at home and hand in to the teacher who helped organize the ex- 
periment, The students were paid when they handed in their completed 
tests. 

Proce du re 

The experimental treatment consisted of a single complete play of 
Ventuie directed by the experimenter and a colleague, followed immediate ly 
by about a half-hour of discussion led by the experimenter. 1 Others pre- 
sent were the experimenter's wife, who assisted in the scoring, and the 
teacher who helped organize the experiment; the letter was preset only 
at the beginning and at the end of Lhe session. Jie session wn ■ he 5 j in 
a classroom in the school building and lasted from about ( J:00 n,m. to 



Appendix A contains an outline of the post-game discussion. 
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about 3:00 p,m,, with two breaks of about a half-hour each, for the 



simulation, the players were divided into four teams: two teams of four 
members and two teams of three members. The simulated industry was 
identified as the detergent industry, and the teams were given fictional 
product names for identification: "Wiz," ’’Snap, 11 "Keen," and "Tuffo," 

Measuring Instruments 

Lncluded in the Venture materials kit is a "business Operations 
Quiz" consisting of ten multiple-choice items. * Each item tests the 
students' knowledge of a particular fact or concept which could be learned 
from Venture . This quiz was used as the measure of factual, information 
for the experiment. 

To measure the student's ability to evaluate business decisions, the 
experimenter constructed a "situation test" which presented the student 
with a summary of the major decisions made by a Hypothetical company. The 
student was to evaluate the decisions in a short essay. Although five 
specific decisions were listed, only three were actually used in scoring 
the essays* 

The students received the two tests stapled together with a cover 
sheet which read: 

This test is a part of a research study. You will not be 
graded on it by your teachers at Cardinal (iibbons. It 
should take you loss than 45 minutes to complete. 

Please do nol discuss this test with anyone until you have 
finished taking it. Please do not change any answers once 
you have finished it. If you don't know an answer, please 
don't look it up or ask someone; just make the best guess 
you can and go on. 



Appendix 15 contains a copy of this test. 
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Answer the business Operations Quiz first: then go or to 
the Situation test. 

The essays were scored by two scorers who had not participated in 
any other aspect of the experiment. The scorers were given a model an- 
swer and a specific scoring formula to use In scoring the essays. ^ The 
essays were presented to each scorer in a different random order, and 
the student's score for the test was the sum of the scores his essay re- 
ceived from the two scorers* 

The school also made available to the experimenter a list of the 

students’ verbal ability scores. The verbal ability scores are grade- 

2 

equivalent scores from the Educational Development Series, taken at 
grade-level 11.3. 



Appendix C contains a copy of the situation Lose, L: r del answer, 
and the scoring formula. 



Published by Scholastic Testing Service, Bens envi lie, Illinois. 
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The distributions of scores on the c^uiz and on the situation test 
are shown in Table 1 below. 



Table l 



Distributions of Scores on 
Multiple-Choice Quiz and Situation Test 



Multiple-Choice Quiz 


Situation Test 




Mo. of Students 


Simu Lat ion 


Control 


No . of Students 


Simulation 


Contro 1 


With Score of: 


Croup 


G roup 


Wi th Score of : 


Croup 


Group 


10 


0 


0 


8 


2 


1 


9 


1 


1 


7 


1 


0 


8 


1 


1 


6 


0 


0 


7 


5 


2 


5 


3 


0 


6 


4 


2 


4 


2 


2 


5 


3 


2 


3 


0 


3 


4 


0 


2 


2 


2 


3 


3 


0 


2 


1 


1 


4 


2 


0 


2 


0 


3 


I 


1 


0 


0 








0 


0 


0 








Total Number of 






To t a 1 Numb e r of 






S t u d e :i t s 


14 


14 


Students 


14 


14 


Mean Score 


6.30 


3.07 


Mean Score 


3.64 


2.30 


Standard Levin 


n 




Standard Deviation 




of Scores 


1.12 


2.12 


of Scores 


2.77 


1 . 94 
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The experimental group outperformed the control group on bc^h tests, 
us predicted. However, the difference between the groups approached sta- 
tistical significance only for the multiple-choice quiz (,05 < p < .10), 

not for the situation test. Significance was tested by the Mann-Whitney 
IJ-test, using a two-tailed test; the values of U were 58 for the quiz and 
78.5 for the situation test. The point-biserial correlation between quiz 
score and experimental treatment was .39, which indicates that the treat- 
ment accounted for about 15 percent of the variance in the quiz scores. 

The correlation between situation tes‘ score and experimental treatment 
was .23; the treatment accounted for only about 5? percent of ihe variance 
in the situation test scores. 

ihe internal consistency of the quiz scores, computed by KR-20 in 
both groups combined, was .52. Ihe inter-scorer correlation for the situ- 
ation test was .75; stepped up by the Spearman-Brown formula with n - 2, 
t lie inter-scorer consistency estimate for the sum of the scores assigned 
by the two scorers becomes .84. (Note, however, that high inter-scorer 
consistency is not a sufficient condition for high reliability; an objec- 
tive test may have very low reliability, but its in ter- scorer consistency 
will always be 1,00,) 

The vtrbal ability scores were collected for use as a control vari- 
able, Unfortunately, although the verbal ability scores were almost 
identical in the two gro’g j twi \ a mean and standard deviation of 12,44 
and .88 for the experimental group as compared with 12,55 and ,84 for the 
control group), the relationship between te^t stores and verbal ability 
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was much weaker in the experimental group. The relationship was aLsc 
much weaker for the situation test than for the multiple-choice quiz. 
Figure 1 shows these relationships in graphic form; Table 2 shows the 
correlation coefficients. 



Table 2 



Inter correlations 


of Scores Within Each Group 






Correlation of . . , 


S imu la t ion 
Group 


Control 

Group 



Quiz with 


Situation Test 


.59 


.56 


Quiz with 


Verbal Ability 


.58 


.69 


Situation 


Test with Verbal Ability 


-.02 


,53 



No verbal ability score was available f^r one of the students in 
the control group. Since he scored 8 on the quiz--nearly three points 
above the mean* -his score might have affected the relationship between 
verbal ability and quiz scores in the control group. However, since 
this student scored 3 on the situation test--very near the mean of the 
control group- -his score would not have had a large effecL on the re- 
lationship between verbal ability and situation test scores. 
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Figure 1 



Scatterplots of Multiple-Choice Quiz Scores 
And Situation Test Scores by Verbal Ability 
(X = Simulation Group; 0 = Control Group) 
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DISCUSSION 



The result's of the quiz show clearly that the simulation was effec- 
tive for teaching farts, especially for students of low verbal ability. 
Students of high verbal ability seemed to know many of the facts that 
were tested, even without receiving any instruction, while students of 
Low verbal ability learned these facts from the simulation. The results 
of the situation test show that simulation was also effective for teach- 
ing decision-making and knowledge of relationships, but this effect is 
much weaker. Thr2e of the students in the simulation group failed to 
mention any of the four specific relationships for which the situation 
test was scored, even though two of these relationships were mentioned 
during the post-game discussion. 



The most puzzling aspect of these results is the relative strength 
of the effects on the two different tests. Advocates of simulations for 
teaching usually emphasize the value of simulations for teaching objectives 
like those tested in the situation test, rather than for teaching facts. 
However, the implied comparison is probably between simulations and other 
methods of teaching, rattier than between simulations and no treatment. If 
the control group in this study had received instruction bv conventional 
classroom techniques, they might have done as veil as the simulation group 
on the quiz, since previous studies have shown simulations and conventional 
classroom techniques to be about equally effective in teaching facts, 

T his study suggesLs-~tentatively--that a simulation can also be at least 
partially effective in teaching decision-making and knowledge of relation- 
ships, Whether simulations are move effective than conventional classroom 
methods for achieving these objectives is a question chat remains unatisvereo. 
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APPENDIX A 



Outline of Foot-Game Discussion 



In directing the post-game discussion, the experimenter . . 

1. Asked the teams to explain their strategy: 

a. Whether they based decisions on own company’s experience or 
anticipating other companies' strategies, 

b. Whether they thought marketing and advertising was impo*. ant. 

c. Whether they thought research and development was important, 

2. Asked the students whether they would play differently if they were 
to play the game again. 

3. Pointed out that high school students in a business simulation are 
less predictable than real business executives, 

A, Explained that, in the game, sales potential depends on both selling 
price and marketing expenditures of one’s own company and of other 
companies, (This was in response to a student’s question,) 

5, Stated that the purpose of the experiment was to find out what the 
game teaches. (Also in response to a student's question.) 

6, Asked the students what they thought they had learned from the game, 

7, Asked what they had expected to learn from the game, 

8, Asked whether they had enjoyed the game. 

9, Requested that the students not discuss the game with other students 
who had not played it, until all tests had been handed in. 
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APPENDIX B 



Business Operation Quiz 



1. The expense incurred by a company as buildings and equipment wear out 
is ca J 1 ed : 

(a) Automation (b) Depreciation (c) Inflation (d) Incapacitation 
(e) Demurrage 

2. Stockholders' dividends are paid from: 

(u) Profits before taxes (b) Profits after taxes 

(c) Operating expenses (d) Working capital 
(e) Plowback tund s 

3. Plowback- -the r einvestniCMit of capital in a business-~may be made only 
with funds taken from: 

(a) Operating expenses (b) Profits before taxes 

(c) Profit after taxes (d) Accounts ’‘eceivable 
(e) None of the above 

4. Among the 300 largest industrial corporations in the U.3.> profits after 
taxes for most of the companies (807,) are likely to be about wh ' per 
cent of sales income? 

(a) 1-87, (b) 11-187, (c) 21-287 0 (d) 33-417, 

(e) 48-557, 

5. Profits are never used for: 

(a) Investments (b) Management salaries (c) Stockholders' dividends 

(d) Increasing production (e) Developing better products 

6. Common stock price reflects the value placed on a company by: 

(a) Banks (b) The government (c) The public (d) Management 

(e) Stock exchanges 

7. A statement of a company's total assets ca nno t include: 

(a) ProducL inventory (b) p rodvition machinery 

(c) Office buildings (d) Sales Income 
(e) Working capital 

8. A company pays federal income taxes: 

(a) On funds left after paying dividends to stockholders (b) On sales 
income less operating expenses (c) On funds added to working cap. tal 

(d) On funds remaining after plowback (e) None of the above. 




is deter- 



9. The amount of dividends a company pays to its stockholders 
mined by: 

(a) law (1 ) Managemert (c) Stock exchanges (d) Stockholders 
(e) Sto'ik t rokers 

10. Sales of a company's products are likely to be influenced l east by: 

(a) Research and development (b) Common stock price (c) Product 
price (d) advertising (e) Production capacity. 
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APPENDIX C 



Situation Test 



Last year the ADC C^mapny was producing it 907, of capacity. Inventory 
levels were constant, and the company showeu a profit. Market research 
shewed that the cotal market potential for the company's product this year 
would be about the same as last year. The production department reported 
that they would be able to keep costs this ^ ear at the same level as last 
year , 



This year the company's management decided to: 

(1) raise the selling price of the company's product; 

(2) increase production to 9 57, of capacity; 

(3) spend less on marketing and advertising; 

(4) spend less on product research and deve lopment ; 

(5) spend more on investment to reduce production costs. 

On the basis of the information you have, do you think these were good 
decisions? Explain why or why r.ot • De as specific as you can. 
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Sarnole Answer for Situation Test 



The. ABC com r my has raise d the price of its product and decr e ased its 
marketing and advertising. Both oi these changes should result in lower 
sales. Yet the company has increased production at the same time* This 
combination of decisions v»ould make sense only if the company's inventory 
levels had been falling drastically ► But inventory levels had not been 
falling at all; they were constant. 

Scoring Formula for Situation Test 

1 point for mentioning each of the following relationships: 

Lower marketing and advertising means less sales. 

Higher price means less sales. 

More production for less sales generally doesn't make sense. 

Constant inventory rules out the possibility that the company was 
trying to avoid running out of product. 



